As action decoder their mentioned architectures of is multimodal adopted in the in to paper Figure information generate, the 1. visual-gr natural with languages cross-attention ounded alignment conditioned blocks, decoder on while the is visual applied the visual-grounded input. Based on these deeply fused representations, we finally generate the predicted answers with the visual-grounded generation decoder. In this section, we describe the settings used when fine-tuning the pretrained models on various downstream tasks. We use RandomAugment [1] for data augmentation. The default settings for finetuning on each dataset are shown in Table 1.

D.2 Countries Hyperparameters are summarized in table 6. We ran all experiments on a single CPU (Apple M2). 15 optimizer AdamW learning rate 0.0003 learning rate schedule cosine training epochs 100 weight decay 0.00001 batch size 4 embedding dimensions 10 embedding initialization one-hot, fixed neural networks LeNet5 max search depth / Table 5: Hyperparameters for the MNIST -addition experiments.

We use the same setting for different sizes RevCol models on MIM pre-training. The hyper-parameters generally follow [4, 2]. Table 3 shows the detail training settings after MIM pre-training. We also show training settings on ImageNet-1K after ImageNet-22K fine-tuning. For semantic segmentation, we evaluate different backbones on ADE20K dataset.